Inferential analysis using feedback for extracting and combining cyber risk information

ABSTRACT

Various embodiments of the present technology include methods of assessing risk of a cyber security failure in a computer network of an entity. Various embodiments also include automatically determining, based on the assessed risk, a change or a setting to at least one element of policy criteria of a cyber security policy, automatically recommending, based on the assessed risk, computer network changes to reduce the assessed risk, and providing one or more recommended computer network changes to reduce the assessed risk. Various embodiments further include enactment by the entity of at least one of the one or more of the recommended computer network changes to reduce the assessed risk to the entity, determining that the entity has enacted at least a portion of the recommended computer network changes, and in response, automatically reassessing the risk of a cyber security failure based on the enacted recommended computer network changes.

CROSS REFERENCE TO RELATED APPLICATIONS

This Non-Provisional Patent Application is a continuation-in-part ofU.S. patent application Ser. No. 14/931,510, filed Nov. 3, 2015 (nowU.S. Pat. No. 9,373,144, issued Jun. 21, 2016), which is acontinuation-in-part of U.S. patent application Ser. No. 14/585,051,filed Dec. 29, 2014 (now U.S. Pat. No. 9,253,203, issued Feb. 2, 2016);this Application is also a continuation-in-part of InternationalApplication No. PCT/US2015/067968, filed Dec. 29, 2015, which is acontinuation of U.S. patent application Ser. No. 14/614,897, filed Feb.5, 2015, which claims the benefit of U.S. Provisional Patent ApplicationSer. No. 62/098,238, filed Dec. 30, 2014. All of the above applicationsare hereby incorporated by reference herein in their entirety includingall references cited therein.

FIELD OF THE PRESENT TECHNOLOGY

The present technology relates generally to systems and methods fordetermining metrics, such as diversity or similarity, between entitiesand the application of those metrics as actionable feedback loops whichcan be used to increase diversity or reduce similarity amongst groups ofentities. These metrics may relate to diversity of aggregate cybersecurity risk for use in planning or filtering new entities so as toincrease that diversity.

SUMMARY

Various embodiments of the present technology include a methodcomprising assessing risk of a cyber security failure in a computernetwork of an entity, using a computer agent configured to collectinformation from at least publicly accessible Internet elements. In someembodiments of the present technology, the assessing of risk includesevaluating the collected information to obtain circumstantial orindirect information that is indicative of the entity and crossreferencing data in the collected information to confirm or infer thatthe entity is referenced in the circumstantial or indirect informationthat is indicative of the entity being referenced in the circumstantialor indirect information. Various embodiments include at least one ofincreasing or decreasing the assessed risk if the circumstantial orindirect information is negative or positive. Various embodiments alsoinclude automatically determining, based on the assessed risk, a changeor a setting to at least one element of policy criteria of a cybersecurity policy, automatically recommending, based on the assessed risk,computer network changes to reduce the assessed risk, and providing oneor more recommended computer network changes to reduce the assessedrisk.

Various embodiments of the present technology further include enactmentby the entity of at least one of the one or more of the recommendedcomputer network changes to reduce the assessed risk to the entity,determining that the entity has enacted at least a portion of therecommended computer network changes, and in response, automaticallyreassessing the risk of a cyber security failure in the computer networkof the entity based on the enacted recommended computer network changes.Various embodiments of the present technology also include dynamicallyre-determining, based on the reassessed risk of a cyber security failurein the computer network of the entity, the change or the setting to theat least one element of policy criteria of the cyber security policy.

BRIEF DESCRIPTION OF THE DRAWINGS

Certain embodiments of the present technology are illustrated by theaccompanying figures. It will be understood that the figures are notnecessarily to scale and that details not necessary for an understandingof the technology or that render other details difficult to perceive maybe omitted. It will be understood that the technology is not necessarilylimited to the particular embodiments illustrated herein.

FIG. 1 is a high level schematic diagram of computing architecture forpracticing aspects of the present technology.

FIG. 2 is a flowchart of an example method for determining entitydiversity.

FIG. 3 is a flowchart of an example action and feedback loop method forupdating a diversity score and improving client diversity.

FIG. 4 is a flowchart of a method for analyzing a new client's impact onan existing diversity calculation.

FIG. 5 is a block diagram illustrating a device according to an exampleembodiment.

FIG. 6 is an example graphical user interface (GUI) that comprises agraphical representation that plots an entity's motivation andsophistication relative to cyber risk.

FIG. 7 is an example graphical user interface (GUI) that comprises ascatter plot illustrating an entity's motivation and sophisticationrelative to cyber risk.

FIG. 8 is an example graphical user interface (GUI) that comprises a bargraph illustrating the plotting of a plurality of entities based ontheir combination scores.

FIG. 9 is an example graphical user interface (GUI) that comprises a bargraph illustrating the plotting of a plurality of entities based ontheir sophistication scores.

FIG. 10 is an example graphical user interface (GUI) that comprises abar graph illustrating the plotting of a plurality of entities based ontheir motivation scores.

FIG. 11 is an example graphical user interface (GUI) that comprises ascatter plot that represents a plurality of entities plotted accordingto their combination score.

FIG. 12 is an example graphical user interface (GUI) that comprises ascatter plot that represents a plurality of entities plotted accordingto their combination score, as well as additional graphicalrepresentations for an entity and a list of recommendations based on theplotting.

FIG. 13 is a flowchart of an example method of the present technology.

FIG. 14 is a flowchart of another example method of the presenttechnology.

FIG. 15 is a flowchart of yet another example method of the presenttechnology.

FIG. 16 is a flowchart an additional example method of the presentdisclosure.

FIG. 17 is a schematic diagram of a computing system that is used toimplement embodiments according to the present technology.

DETAILED DESCRIPTION

Various embodiments of the present technology are directed to systemsand methods for determining diversity and/or similarity between entitieswith respect to risk, (e.g., cyber security risk), and the utilizationof these metrics in various ways to improve diversity between theanalyzed entities. In one embodiment, an insurer may desire tounderstand the diversity of their insured entities with respect toaggregate cyber risk and utilize a measure of diversity to prevent toomuch similarity between insured entities, and/or to compare theirdiversity to their industry peers. Additionally, reinsurers, ratingagencies and/or insurance brokers may also utilize the presenttechnology. For example, reinsurers may want to compare one insurer'sportfolio to another insurer's to buy, invest, and/or cover. Brokers maywish to review their portfolio of clients, and ratings agencies mayreview an insurer's portfolio and use it to provide a rating on thefinancial strength rating of the insurer. To be sure, cyber insuranceand other insurance risks can be a function of similarity. For cyberinsurance risk, if insured entities are very similar to one another in avariety of key attributes such as revenue, clientele, industry,technology utilized such as cloud computing service provider, contentdelivery network (CDN) provider, operating system, firewall vendor,intrusion detection system vendor, security services provider, etc., orother factors, a loss, (e.g., due to a cyber attack), by one of theseinsured entities might imply that other insured entities having similarattributes will also experience a loss. For example, a plurality of webhosting provider may source their servers from the same company. A cyberattack of that company's servers may equally affect all of these webhosting providers that use the server, and consequently affect aninsured that utilizes one of those web hosting providers to host theinsured's website and other web services.

To be sure, diversity in attributes between entities can decrease thelikelihood that a covered loss by any particular entity will also likelyaffect the other entities. Thus, the desire is to have the insuredentities be as diverse as possible in the aggregate, to reduce overallrisk. Conversely, similarity of attributes between insured entities canincrease risk for the insurer.

Using the present technology, an end user may determine similarattributes shared between pluralities of entities. These sharedattributes can be aggregated into clusters to locate groups of entitieswith shared attributes. In one example, several entities use the samecontent delivery network (CDN), the same cloud service provider, asimilar website traffic profile, have overlapping executives, and reportsimilar revenue. While these entities may also share attributes withother entities, these attributes are used in various embodiments tocreate a cluster or grouping of entities that, when considered in theaggregate, have a low diversity score due to the similarities in thisexample.

End users may use the present technology to learn their aggregate cyberrisk compared to industry peers and use that information to, forexample, screen potential target entities for inclusion into a groupbased upon how the potential addition of their attributes to the groupwould affect the diversity score for the end user's collection ofentities. In alternative exemplary embodiments, the system may insteadof, or in addition to, outputting a diversity or clustering score, mayoutput a different value analyzing the entities, for example a probablemaximum loss (PML) and/or an expected portfolio value.

The present technology can be used to analyze diversity/similaritybetween many entities. The diversity/similarity analyses can usehundreds and even thousands of attributes, looking for diversity orcommonality therebetween. In some instances, the end user can adjust theattributes and/or select which attributes are important to them and thesystem will analyze only these attributes when determining diversity,(e.g., a diversity score for aggregate cyber risk).

While the examples above mention the suitability of the presenttechnology for use with insurance planning, in general, and cyberinsurance planning, in particular, the present technology is not solimited. Other examples of technologies that can implement the presenttechnology are financial portfolio managers, technology companies thatdesire infrastructure robustness, human resources, venture capitalinvestment, and so forth.

These and other advantages of the present technology are provided belowwith reference to the collective drawings.

FIG. 1 is a high level schematic diagram of a computing architecture(hereinafter architecture 100) of the present technology. Thearchitecture 100 comprises a diversity analysis system 105 (hereinafteralso referred to as system 105), which in some embodiments comprises aserver or cloud-based computing device configured specifically toperform the diversity analyses described herein. That is, the system 105is a particular purpose computing device that is specifically designedand programmed (e.g., configured or adapted) to perform any of themethods described herein.

The system 105 can be coupled with end user device 105A, such ascomputer, tablet, Smartphone, or other similar end user computingdevice. End users can interact with the system 105 using their end userdevice 105A. The end user device 105A and system 105 can be coupledusing a network 105B.

A suitable network 105B may include or interface with any one or moreof, for instance, a local intranet, a PAN (Personal Area Network), a LAN(Local Area Network), a WAN (Wide Area Network), a MAN (MetropolitanArea Network), a virtual private network (VPN), a storage area network(SAN), a frame relay connection, an Advanced Intelligent Network (AIN)connection, a synchronous optical network (SONET) connection, a digitalT1, T3, E1 or E3 line, Digital Data Service (DDS) connection, DSL(Digital Subscriber Line) connection, an Ethernet connection, an ISDN(Integrated Services Digital Network) line, a dial-up port such as aV.90, V.34 or V.34bis analog modem connection, a cable modem, an ATM(Asynchronous Transfer Mode) connection, or an FDDI (Fiber DistributedData Interface) or CDDI (Copper Distributed Data Interface) connection.Furthermore, communications may also include links to any of a varietyof wireless networks, including WAP (Wireless Application Protocol),GPRS (General Packet Radio Service), GSM (Global System for MobileCommunication), CDMA (Code Division Multiple Access) or TDMA (TimeDivision Multiple Access), cellular phone networks, GPS (GlobalPositioning System), CDPD (cellular digital packet data), RIM (Researchin Motion, Limited) duplex paging network, Bluetooth radio, or an IEEE802.11-based radio frequency network.

In one embodiment, the system 105 comprises a processor 110 and memory115 for storing instructions. The memory 115 can include an attributemodule 120, a comparator module 125, a clustering module 130, aweighting module 135 and a recommendation module 140. As used herein,the terms “module” may also refer to any of an application-specificintegrated circuit (“ASIC”), an electronic circuit, a processor (shared,dedicated, or group) that executes one or more software or firmwareprograms, a combinational logic circuit, and/or other suitablecomponents that provide the described functionality.

For context, the diversity analyses according to various embodiments ofthe present technology begin with input for the attribute module 120. Aset of variables that are indicative of attributes of an entity may beinput into the attribute module 120. In one embodiment, the variablescan include technologies a company might employ (e.g., internally andexternally for Internet communication such as e-mail, website, andsocial media online presence) such as CDN provider, cloud serviceprovider, server type, OS type, visitor traffic knowledge, customerprofiles, as well as other non-technical information such as revenue,number of employees, years in business, and so forth. In variousembodiments, the breadth and type of variables that can be analyzed andcorrelated are unlimited. In some embodiments, the breadth and type ofvariables that can be analyzed and correlated for the company and fortheir industry peers, for comparison, may be limited by breadth and typeof information that is available at online sources concerning the same.Again, an end user can define or specify the types of variables that areof interest to them.

For example, if the end user is an insurer, the insurer may desire toknow how diverse their insured entities are with respect to cybersecurity risk relative to a wide and divergent set of variables. Inregard to a cloud computing provider, for example, interest in suchdiversity may be only in technological variables such as traffic, pageviews, bandwidth, and other variables related to cyber risk.

In some embodiments, entities and end users can access and interact withthe system 105 using a variety of graphical user interfaces (GUIs) suchas a dashboard, including various elements as described herein. Thesystem 105 can use the dashboard to display messages or notifications aswell as diversity scores, similarity scores, and/or recommendations.

The system may gather variables for an entity by querying the entity forinformation, scraping available online sources such as websites,corporate filings, news sources, other public record databases andresources. Additionally, data may be gathered from the entity's networkusing devices already present there or by placing a new device on theentity's network to gather more data. The data collecting device may bea server, router, firewall, switch, or repeater, or may be a softwareagent or routine that monitors traffic and/or performs packetinspection. The data collecting device may be on the company's networkand/or its periphery, and may collect and/or analyze the data, whilealso transmitting it to system 105. In this manner, additional,proprietary data may be gleaned from a particular entity's network.Regardless of how the variables are obtained, the variables are inputinto the attribute module 120. The attribute module 120 can format ornormalize the input as needed for consistency.

In one embodiment, the comparator module 125 is executed to perform avariable comparison on all or a subset of the variables. The comparisoncan be for all or only a subset of all entities. The subset of variablescan be selected by the end user, as well as the entities analyzed.

The comparator module 125 is configured to identify variables sharedbetween entities or groups of entities. The implications of thisanalysis are multifaceted. For instance, the same variable can be sharedbetween many entities, which leads to an inference that a particularvariable might be problematic. This lack of diversity is a more pointedor granular lack of diversity.

In another example, multiple variables are shared between numerousentities. This diversity relationship between the entities signifies amore prolific lack of diversity.

Localized commonality can be found between small groups (even betweentwo) entities. This type of similarity can be inferred as lessproblematic than the more prolific examples provided above wheresimilarity exists between numerous entities.

The comparator module 125 can cooperate with the clustering module 130to create commonality clusters (e.g., various clusters of commonlyshared variables). In one embodiment, if five entities are beinganalyzed, many different clusters can be identified. By example, ifvariables A-D are being analyzed with respect to entities 1-5, thecomparator module 125 finds commonality between entities 1 and 3 withrespect to variables B and C. Also, the comparator module 125 findscommonality between entities 1-5 with respect to variable A. Othersimilar correlations can be found.

The clustering module 130 can display to the end user these commonalityclusters, which indicate areas of non-diversity. Also, these commonalityclusters can be utilized by the recommendation module 140 to createaction items for the end user that if enacted would change the diversityscore. Details regarding the diversity score are found in greater detailbelow.

In some embodiments, the comparator module 125 creates a diversity scoreor index. This diversity score represents how dissimilar the analyzedgroup of entities is relative to one another in view of their variables.

The diversity score can include a percentage of the overall number ofcompared variables that are dissimilar to those that are shared. Thediversity score can be represented variously as a fraction, a decimal,or a percentage, and may be included in the graphical user interface(e.g., dashboard.) Additionally, or alternatively, the diversity scoremay be normalized into a number within a user-defined, or predefined,range, similar to a credit score.

In some embodiments, the comparator module 125 can cooperate with theweighting module 135 to applying a weighting to one or more variables.In one embodiment, the weighting is selected by an end user such as aninsurer. For example, an insurer determines that industry serviced,gross revenue, and customer country of origin are important variables toanalyze, (e.g., for assessing individual and aggregate cyber risk.) Forinstance, if the insurer knows that gross revenue is very important tothe calculation, the insurer can specify that the gross revenue variableis to be given greater importance in the analysis than other variables.In another example, the insurer can assign a weight to each variablebased upon importance.

In some embodiments, the system can determine weightings and variablesbased on industry knowledge acquired, and use machine learning, big dataand other “tools” to make an “educated” determination. For example, theweighting of variables can also be determined by the system 105 based oninformation such as actuarial data, industry practices, or other rulesestablished by end users but which are intended to be applied bydefault. The selection of a weighting schema by the system 105 can bebased on variables for the entities. For example, if the system 105determines that the entities are all physicians, the system 105 canselect weightings that are appropriate for medical practices orhospitals. Such determinations by the system may be adjusted and/orotherwise specified by the end user (e.g., using the dashboard) totailor them for their particular circumstances, preferences, or otherfactors.

In some embodiments, the diversity score can be represented as adiversity graph that illustrates the connection between entities.Entities can be graphically connected based on commonality of variablesbetween entities. For example, certain entities may be connected asbeing banks that present particularly enticing targets for cybercriminals and thus particular cyber risks.

In response to calculating a diversity and/or similarity score, therecommendation module 140 can be executed to provide the end user withsome type of actionable feedback. For example, the recommendation module140 can provide the end user one or more actions to the end user basedon the diversity score and the clusters of similar variables. These oneor more actions potentially increase the diversity score if enacted bythe end user.

In one example, the recommendation module 140 can automatically identifyvariables, which if changed, would affect the diversity score. Forexample, if the entities are or utilize technology company serviceproviders that use a particular CDN, the recommendation module 140 canoutput a recommendation that diversification in this area would bebeneficial. The end user can alert the entities and encourage them toexplore other options for CDNs. If the end user is an insurer, forexample, the insurer can encourage this change by offering rateconcessions to the insured entities. Various embodiments of the systemthus may automatically provide the diversity score or other informationto the end user regarding diversity, which the end user can utilize toencourage or effect various changes (e.g., via rate concession,screening of potential new entities, adjusting rates based on diversity,or other actions prompted by the system's determinations.) The diversityscore might also be used to inform the insurer as to which policesshould be renewed and which polices should not be renewed. For example,if a potential new (target) entity to add presents an unacceptable cyberrisk, based on the diversity analysis, the insurer may choose not toprovide the entity′ policy or to provide the policy at a ratecommensurate with the risk.

In another example, the recommendation module 140 can identifyproblematic common variables that negatively impact diversity scores.For example, the recommendation module 140 may identify sharedinfrastructure such as CDNs and cloud service providers as particularlyproblematic variables that are commonly shared between several entities.In some embodiments, the recommendation module 140 can also identifynetwork traffic, network traffic patterns, firewalls, firewall policiesthat are commonly shared. Changing these shared variables would likelyincrease the diversity score for these entities. Conversely, therecommendation module 140 can determine key variables that if changedwould negatively affect a diversity score. The recommendation module 140can identify these variables to the end user as desirable.

Actions that could be taken in response to this information couldinclude a project plan that specifies that the insurer is to find newcustomers that do not share these problematic variables. Likewise, theproject plan could also or alternatively specify that the insurer is tofind new customers that do share key positive variables.

In one example, an action includes the recommendation module 140creating and providing the end user with a variable profile of a targetentity that when added to the plurality of entities increases thediversity score. For example, the recommendation module 140 could createa profile for a prototypical new client that is in a differenttechnology sector or a completely different industry sector. In anotherembodiment, the recommendation module 140 could create a profile for aprototypical new client that includes desirable variables, rather thanmerely a client that excludes certain disfavored variables.

In one embodiment, the recommendation module 140 can provide the enduser with a list of entities of the plurality of entities that arelowering the diversity score. Again, as mentioned above, certainclusters of variables may be found in common between entities. Certainones of these clusters may have more of a negative impact on thediversity score than others. For example, commonality betweenheadquarters or domicile may have no impact on the diversity score, evenif this variable is shared in common between several entities. On theother hand, commonality in gross revenue or average employee age mayhave a drastic impact on the diversity score for one reason or another.To be sure, commonality of a variable(s) does not always negativelyaffect the end user or the end user's business. In these instances thecommonality can be ignored or weighted so as not to affect thecalculated diversity score.

In another example, the recommendation module 140 can provide the enduser with a list of entities of the plurality of entities that, if lostwould lower the diversity score, which can prompt the end user to takeaction to avoid.

In another example, action the recommendation module 140 can compare avariable profile for a new entity to determine if the addition of thenew entity to the analysis will negatively or positively impact thediversity score of the group.

For example, the attribute module 120 can receive a variable profile fora new entity and parse out the variables which are indicative ofattributes of the new entity. This profile could include an applicationform, a survey, or any other content that is capable of conveyingvariables.

Next, the comparator module 125 adds a set of variables of the newentity to the comparison described above and repeats the calculation ofthe diversity score. The recommendation module 140 can alert the enduser if the addition of the new entity decreases the diversity score.The recommendation module 140 can alert the end user if the addition ofthe new entity increases the diversity score as well.

In some embodiments, the recommendation module 140 updates the diversityscore based on feedback received from the end user. For example, if theend user wants to view how the addition of a proposed new client willaffect an existing diversity score, the profile for the new client isadded to the system and the variables for the new client are processedand added to the comparison process. A new or updated diversity score iscalculated and displayed to the end user.

The difference between the new diversity score and the old diversityscore is expressed as a diversity delta. In some embodiments, the system105 can apply thresholds to the diversity delta to determine if aproposed change to the entity grouping is sufficient to warrant theproposed change. For example, the system 105 may require at least a netchange or diversity delta of 20%. Other percentages can also beutilized. The present technology provides information related to theupdated information (the new diversity score), including differences(the amount of the change made in one or more updates, namely thedelta), and trends (patterns over many time steps).

The present technology also provides attribution information when adiversity score changes. In particular, the methods and system indicateto a user why the score has changed, namely what exactly has changed inthe underlying data sets to effect that higher level score change. Inthis manner, the systems and methods of the present technology providedetailed information to the user to identify the changed data, andthereby understand the positive and negative impacts of the user'sactions on the diversity score.

The system 105 can also build an entity portfolio for an end user withknowledge gained from an analysis of variables for a plurality ofentities. For instance, the system 105 can create a report that informsthe end user as to how many and what type of entities a portfolio shouldhave to be balanced in terms of diversity, (e.g., with respect to cyberrisk.) For example, the report may indicate that an insurer should havea certain percentage of clients in the banking sector, a certainpercentage in the technology sector, and a certain percentage in themedial industry. These sectors of the portfolio are deduced by comparingvariables for various entities in a given industry that lead to asuitable diversity score.

It will be understood that the diversity score can be counterbalanced byother factors such as revenue for the end user. That is, an insurer maybe more likely to accept a lower diversity score from a group ofentities that pay higher premiums or a group of entities that is atleast partially self-insured.

FIG. 2 is a flowchart of an example method 200 that is executed by thesystem (e.g. system 105), in accordance with the present technology. Themethod 200 includes the system 105 (for each of a plurality ofentities), receiving 205 a set of variables that are indicative ofattributes of an entity. These variables can include any number or typeof variables that represent the attributes of the entity.

These variables are collected for numerous entities that may belong, insome embodiments, to a particular class or group. For example, theentities could include all employees in a company, all insured customersof an insurance agency, investors in a mutual fund, or other groups.

Next the method 200 includes the system 105 comparing 210 the sets ofvariables for the plurality of entities to each other, and locating 215clusters of similar variables shared between two or more of theplurality of entities.

Next, the method 200 includes the system 105 clustering 220 commonvariables and identifying the entities that share the common variables.These clusters are indicative of non-diversity between these entities.

Next, the method 200 includes the system 105 calculating 225 a diversityscore that represents how different the plurality of entities are to oneanother based on variables that are not shared between the plurality ofentities. This diversity is directly related to the commonalitydiscovered above. The more similar or commonly shared variables exist,the less diverse the entities are relative to one another, as a generalrule. Again, as mentioned above, some variables will have little to noimpact on diversity as dictated by weighting or variable selection bythe end user. For example, if a commonly shared variable is not includedin the diversity calculation by the end user the variable has no impacton the diversity score.

Next, the method 200 includes the system 105 receiving 230 feedback froman end user in response to providing the diversity score to the enduser. Also, the method 200 includes the system 105 updating 235 thediversity score in response to the feedback.

Various types of feedback are contemplated and illustrated in FIG. 2.The feedback can take the form of a suggestion, option, report, or otheroutput that is actionable by the end user. Exemplary methods and systemsaccording to the present technology may also provide benchmarking overtime. In this manner, an insurance company or other entity trackingaggregate cyber risk may track their diversity score over an adjustabletime period, for example days, weeks, months, and/or years.

It will be understood that the methods illustrated in flowchart form aresusceptible to execution in various forms such that not all steps may berequired. In some instances, additional steps can be added. Some stepsmay be rephrased or replaced with other steps, in accordance with theclaimed technology.

In FIG. 3, the flowchart illustrates the method 300 including the system105 providing 305 the user with one or more actions/suggestions that tothe end user based on the diversity score and the clusters of similarvariables. These actions can potentially increase the diversity score ifenacted by the end user.

In step 310, an action includes providing the end user with a variableprofile of a target entity that when added to the plurality of entitiesincreases the diversity score.

In step 315, an action includes providing the end user with a list ofentities of the plurality of entities that are lowering the diversityscore.

In step 320, an action includes providing the end user with a list ofentities of the plurality of entities that, if lost, would lower thediversity score.

Regardless of the action taken (steps 310, 315, and/or 320), thefeedback is used in calculating 325 an updated diversity score anddelivering 330 the updated diversity score to the end user.

Again, these options are merely examples and are not intended to belimiting. These options can be provided individually or in combination,if desired.

FIG. 4 is a flowchart of a new entity analysis method 400. In the method400, the system (e.g. system 105) is utilized to compare the variablesof a new entity to an existing diversity analysis. For example, aninsurer desires to determine how the addition of this new entity willaffect the diversity of an existing client base. This aggregate riskanalysis can be used to ensure that diversity is maintained or increasedwhen a new client is added to an existing pool of clients.

The method 400 includes receiving 405 a variable profile for a newentity. The variable profile either includes a set of variables or a setof variables is deduced from the variable profile. As mentioned above,the variable profile can include an application form, a resume, acorporate filing such as a tax return, or any other document thatincludes attributes of an entity.

Next, the method 400 includes adding 410 the set of variables of the newentity to the variables of the previously analyzed entities andperforming 415 an updated comparison of variables. Next, the method 400includes generating 420 an updated diversity score calculation.

In some embodiments, the method 400 includes alerting 425 the end userif the addition of the new entity decreases (or increases) the diversityscore. The end user can decided to accept or reject this new clientbased upon how the client affects the diversity score.

Advantageously, the present technology can be used in scenarios wherediversity of clientele is desirous. The present technology can performdiversity analyses on potentially thousands of attributes acrosscountless entities in ways that would be impossible to accomplish absentthe use of the diversity analysis system. The diversity analyses of thepresent technology can bring clarity to business planning and projectmanagement, where integration of new clients/entities may affect thediversity of a current client base (either positively or negatively).Where diversification is desirable or required, the present technologyprovides a means for facilitating and maintaining this diversity in away that is actionable and usable to the end user. That is, the presenttechnology provides a way for end users to mitigate risk throughdiversification of their customer base or however diversity impactstheir particular business or operations.

Various embodiments of systems and methods are provided for assessingand reducing cyber risks associated with companies or other entities. Invarious embodiments, a method comprises assessing risk of a cybersecurity failure in a computer network of an entity, using a computeragent configured to collect information from at least publiclyaccessible Internet elements. The cyber security failure may include acyber attack and/or a privacy incident (including but not limited to anincident involving sensitive information), to name just a few. Thecomputer agent may be further configured to collect and/or analyzeinformation from the computer network of the entity.

The exemplary method includes automatically determining, based on theassessed risk, a change or a setting to at least one element of policycriteria of a cyber security policy. The cyber security policy may be apolicy from an insurance company, a product warranty for first and/orthird party costs that an entity purchases from one of a networking,security product, or services provider, to name a few. In variousembodiments, the method includes automatically recommending, based onthe assessed risk, computer network changes to reduce the assessed riskand providing one or more recommended computer network changes to reducethe assessed risk, enactment by the entity of at least one of the one ormore of the recommended computer network changes to reduce the assessedrisk to the entity. In some embodiments, the exemplary method includesdetermining that the entity has enacted at least a portion of therecommended computer network changes, and in response, and automaticallyreassessing the risk of a cyber security failure in the computer networkof the entity based on the enacted recommended computer network changes.The exemplary method further includes dynamically re-determining, basedon the reassessed risk of a cyber security failure in the computernetwork of the entity, the change or the setting to the at least oneelement of policy criteria of the cyber security policy. In variousembodiments, the at least one element of policy criteria of the cybersecurity policy is a term and/or a condition. For example, a term and acondition may include a retention amount, a deductible, a premium, acoverage limit, a future valuation, a term length, and so forth.

Various embodiments of systems and methods are provided for calibratingcyber risks associated with commercial markets. In various embodiments,a method comprises assessing the risk of a cyber security failure in acomputer network of a commercial market, using a computer agentconfigured to collect information from at least publicly accessibleInternet elements. The cyber security failure may include a cyber attackand/or a privacy incident (including but not limited to an incidentinvolving sensitive information), to name just a few. The computer agentmay be further configured to collect and/or analyze information from thecomputer network of the commercial market. In some embodiments, thecommercial market is insurance, corporate credit, small business loans,global equities, derivatives, or futures, to name just a few. In variousembodiments, the policy is a policy from an insurance company, or awarranty, to name a few. In various embodiments, the method includesautomatically recommending computer network changes to reduce theassessed risk; and automatically reassessing the cyber risk of thecomputer network of the commercial market based on the recommendedcomputer network changes. In some embodiments, the entity is acommercial market for at least one of insurance, corporate credit, smallbusiness loans, global equities, derivatives, and futures. In variousembodiments, the method includes calibrating cyber risks associated withthe commercial market.

In some embodiments, the method includes, based on the assessing of riskof the cyber security failure in the computer network of the entity,plotting one or more features of the entity and other members of a peergroup of the entity, the plotting being configured to visuallyillustrate the risk of a cyber security failure in the computer networkof the entity; and the automatically recommending of computer networkchanges being based on the plotting. The plotting may be configured tovisually illustrate the cyber risk of the entity.

In some embodiments, the method comprises determining a sophisticationscore of the entity with respect to cyber risk, which may be consideredthe quality of the defense with respect to repelling, defeating, orpreventing a security failure. Sophistication scores may also bereferred to as susceptibility scores.

In some embodiments, the method also comprises determining a motivationscore of a hacker or other actor with respect to initiating one of acyber security failure. A composite score may be created from themotivation score and the sophistication score.

The exemplary method and system may be used in a cyber policy market,and/or by a cyber policy provider providing policies. The cyber policymay include a cyber risk assessment/management service, which mayprovide feedback to one or both of the policy company and the insuredentity, enabling the entity to determine how to reduce their cyber risk,and/or how they are positioned within their peer group and/or within auniverse of companies with respect to their cyber risk. As used herein,the policy, including but not limited to a cyber policy, may be a policyfrom an insurance company or it could be a product warranty for firstand/or third party costs that an entity purchases from a networking orsecurity product or services provider.

Additionally, following the recommendations may enable the policycompany to update and/or change policy criteria of a cyber securitypolicy. In still further alternatives, the composite score of several ormany entities may be aggregated and used by insurance companies,reinsurance companies, brokers and/or ratings agencies to understandand/or evaluate an aggregate risk and assess insurance premiums and/orreinsurance treaties and/or change or evaluate a credit rating. This isdescribed in further detail above.

Also, as mentioned above, the composite store can include facets ofcreditworthiness, small business risk, and other commercial metrics ofcommercial risk.

Cyber insurance insures entities against damage and/or loss due tosecurity failures (e.g., a cyber attack, a privacy incident). Assessingcyber risk can be a difficult task due to the volatility of the cyberenvironment. For example, a risk of a security failure such as a cyberattack lacks actuarial data since there is an active adversary behindcyber attacks, and past cyber attacks do not predict future cyberattacks. Better analysis of cyber risk, including the risk of securityfailures, and providing greater service to insurance companies andinsured entities, is desirable

The technology disclosed herein provides a cyber risk assessment, andprovides methods and systems for improving a cyber risk assessment, by,for instance, reducing a risk of a cyber attack, predicting theprobability of a cyber attack, and/or determining the extent to which acyber attack might cause damage. Exemplary methods plot the cyber riskwithin a peer group, which may be defined by industry, revenue, and/orany other appropriate metric. Various exemplary methods plot the cyberrisk within the universe of companies, (e.g., universe of companies forwhich such cyber risk has been assessed. Exemplary methods assess riskin a plot using one feature. In other examples, multiple features may beplotted into a matrix.)

For those exemplary matrix embodiments, the assessment of risk isplotted with a two (or more) dimensional analysis, which may be plottedinto a two by two matrix or graph, or in any appropriate alternativevisualization method, particularly for greater than two dimensions. Forexample, the two dimensions may be characterized as 1) motivation (whichmay be synonymous or similar to offense, e.g., the motivation of a badactor to attack an entity) and 2) sophistication (which may besynonymous or similar to defense, e.g., the sophistication of an entityto prevent and/or repel a cyber attack, or compel more responsiblebehavior from employees and associates to prevent a privacy event withrespect to sensitive information). Alternative axes for the twodimensional analysis are also possible, for example, measurements otherthan motivation and sophistication. The system may output an estimated(or expected) commercial impact, which may encompass both the risk of acyber attack, and the potential amount of damage caused by a cyberattack.

In addition to analyzing the cyber risk, the present technology mayprovide enhanced value by quantifying a cyber risk, thereby creating amarket for it. Additionally, the present technology may provide a cyberrisk management service tied to a cyber policy. A cyber policy (alsoreferred to as a cyber insurance policy) as used herein includes anyinsurance policy covering any loss arising out of a security failure,including tangible and intangible property. The policy may cover bothfirst party and third party losses arising out of any perils including asecurity failure. The policy may cover business interruption, loss ofincome, Director and Officer liability, information asset coverage, andextra expense coverage, or any other insured loss arising out of asecurity failure. A cyber policy as used herein includes security andprivacy coverage, including regulatory coverage (e.g., FTC, HealthInsurance Portability and Accountability Act (HIPPA)) covering fines andpenalties, and defense costs and damages. The coverage provided by acyber policy as used herein may provide for privacy breach coaches,forensic experts, a public relations campaign, cyber extortion,information asset recovery, business interruption (including forexample, lost income, extra expenses, and/or all costs incurred but forthe cyber security failure), or any other covered costs or losses.

Aspects of a cyber policy may be altered based on use of, andimplementation of recommendations provided by, the cyber risk managementservice. These aspects may include any policy criteria of the policy.Elements of policy criteria include, for example, a retention amount, adeductible, a premium, coverage limits, future valuation, term length,or a term or condition of the policy.

The analysis may be a position on a graph, and may include a scatterplotof the peer group members, and/or a simple ranking amongst the peers.The analysis may be two (or more dimensional). Additionally oralternatively, the analysis may be resolved into a single compositescore embodying the analysis. The plot may be changed to include more orfewer members of the peer group based on further variables of the peergroup members, for instance, revenue, etc. The plot may include pointsfor a universe of companies along with the points for the particularentity. For a two dimensional analysis example, each axis may be afunction of many sub-variables, discussed herein as examples ofmotivation and sophistication. The sub-variables may be weightedequally, or differently, and the weighting may be static, dynamic, orcustomizable based on different analysis goals. Examples of motivationand sophistication elements will be described in greater detail below.

The exemplary assessment system may provide recommendations to an entityto improve their cyber risk assessment, by, for instance, reducing theircyber risk. This may be accomplished by various methods, includingincreasing the sophistication of the organization or entity, ordecreasing the motivation of the attacker to go after this organizationor entity. The recommendations may be specific and may impact one orboth of the axes of the two dimensional risk analysis. Implementing therecommendations, which may be accomplished in some embodimentsautomatically, may reduce the risk of a cyber security failure.

Implementing the recommendations may impact an entity's relativeposition in their peer group, in a universe of companies, as well as anyexpected commercial impact of a security failure (e.g., a cyber attack,a privacy incident). Additionally, factors beyond the control of thecompany or entity, for instance, actions by the other peer groupmembers, activity in the hacker community or vulnerabilities in softwareand/or hardware, may also impact both a relative risk analysis (e.g.,impacting the company or entity's position in their peer group) and/oran absolute expected commercial loss. This change over time may beaccessible and/or charted for trending information, which may be usefulfor planning and/or changing policy criteria (including the premium) forthe policy. An entity may make a judgment of which recommendations toprioritize in implementation based on the different recommendationsprovided by the system to other members of their peer group. Examples ofrecommendations are illustrated in FIG. 12.

In some embodiments, the recommendations generated for an entity can bechanged in comparison with other entities in a group. Thus, the system505 can provide a first set of recommendations based solely on themotivation and/or sophistication (e.g., cyber risk) analysis for theentity.

In another example, the system 505 can generate a second set ofrecommendations based on a comparison of the cyber risk for the entityto the aggregate risk score for many entities. This second set ofrecommendations includes additional recommendations for the entity whichare determined to improve the cyber risk of the entity.

In some embodiments, the system 505 can determine risk factors that arediscrepant between the entity and another entity (or an aggregate groupof entities) and highlight these recommendations as being unique for theentity. For example, if the entity is the only one out of a group oftheir peer entities that does not use a CDN (content delivery network),the system 505 can highlight this difference. These unique discrepanciescan illustrate areas where the entity is particularly or uniquelyvulnerable.

Stated otherwise, the system 505 identifies clusters of sophisticationelements or motivation elements that are shared between two or more ofthe portfolio of entities. The clusters of sophistication elements ormotivation elements being associated with an increase in cyber risk. Therecommendations generated by the system 505 for an entity of theportfolio of entities will cause a decrease in the cyber risk ifimplemented.

In various embodiments, where scores are tracked over time, the system505 can also be configured to periodically reassess the cyber risk of anentity. In some embodiments, the reassessment occurs after the entityhas implemented one or more of the recommendations.

It may be advantageous for the entity to understand not only that aparticular score was changed, but also what affected the change inscore. Thus, the system 505 is configured to provide attribution for ascore change, including verifiable data including time and attributioninformation. This attribution identifies/represents the underlying dataset which affected the score change, and shows why, how much, and howthe score changes.

By way of example, the entity, unbeknownst to them, has a dramaticincrease in pageviews on their website. This increase in pageviewscauses an increase in the motivation score for the entity. That is, theincrease in pageviews indicates that a hacker might be more motivated tohack the entity's webpage because of its high traffic profile.

In some embodiments, the system 505 can be used to automaticallyinstitute changes on behalf of the entity that will decrease thelikelihood that the entity will experience or be adversely affected by asecurity failure such as a cyber attack. These automatic changes occurbased on the recommendations generated for the entity.

In one example, the system 505 can establish new content hosts for thecontent of the entity. The system 505 can inform the entity thatdiversity in content hosting can decrease the likelihood that all of theentity's content or user information will be exposed, as compared to ifthe content is stored in one centralized location. To be sure, thesystem 505 can be used to automatically change technical aspects of theentity, such as computing diversity, content distribution and delivery,and other technical attributes.

In some embodiments, the system 505 comprises a commercial estimatormodule 550 that is configured to estimate a commercial impact to theentity for a simulated security failure (e.g., a cyber attack, a privacyincident). Thus, the system 505 can execute theoretical or simulatedsecurity failures against a cyber profile of an entity. In one example,the cyber profile for an entity is determined from the varioussophistication and motivation elements determined for the entity. Thecommercial estimator module 550 then calculates the effect of, forexample, a distributed denial of service attack (DDoS) on the entity. Tobe sure, the simulated cyber attack in this example tests thesophistication of the entity and is affected by the motivation regardingthe entity. The economic impact can include an economic impact to theentity itself, other entities that depend upon the entity, orcombinations thereof. For example, a cyber security failure for acommercial institution, such as a DDoS attack, can cause direct economicimpact on the institution from website downtime. The cyber securityfailure can also cause a commercial impact to the customers of thecommercial institution if social security numbers, account numbers, orother sensitive consumer and/or personal information is stolen.

Additionally, implementing the recommendations, provided by the cyberrisk management service for example, may be paired with changes to thepolicy criteria of a policy. For example, implementation of certainrecommendations may be paired with automatic renewal, a consequent lower(or higher or otherwise changed) cyber risk policy premium, bettercoverage limits, better term length, future valuation and the like. Forexample, the change to the policy criteria of the policy may beimplemented after the end of the term (e.g., 1, 3, 6 or 12 months, orany other appropriate term) of the current policy, or may trigger arenewal option at the lower premium rate immediately or on anaccelerated basis. In this manner, a cooperative and constructiverelationship may be achieved between insurers and insured-entities,thereby creating a positive feedback loop of improved cyber preparednessand lower/higher/changed premiums. As discussed previously,implementation of recommendations provided by the cyber risk managementservice may cause a change in any element of the policy criteria of acyber policy. For example, if the sophistication of the entity is low, ahigher deductible may be required, and vice versa. Additionally oralternatively, the type of coverage, a pricing or re-pricing, the amountof limits, an automatic renewal, and/or a renewal commitment, may changebased on an increase or decrease in sophistication of the entity, and/oran increase or decrease in motivation of an attacker of the entity.Additionally, as recommendations are implemented, or other changes inthe entity or the entity's situation, the motivation and sophistication,or other metrics, may change, and consequently a new analysis may beprovided including new and/or changed recommendations for the entity.

Additionally or alternatively, the policy criteria of the policy itselfmay determine and/or change the weighting used in the system 505. Instill further embodiments, a policy may affect the system 505 in otherways. In other words, the policy criteria of a policy may impact anassessment of a cyber risk, and/or an assessment service. For example,if a policy has a high deductible, the assessment service may not assessa motivation to initiate a security event. Various other options forhaving the policy criteria of a policy drive the type of assessmentconducted are also possible.

The cyber risk management service as provided herein may includesubjective evaluations, and may include vulnerability assessments,penetration testing, tabletop exercises, people services, riskengineering, and/or training exercises. Changes or renewed evaluationsof any of these assessments may cause an increase or decrease in asophistication of the entity, an increase or decrease in a motivation ofan attacker of the entity, and/or a change in any other metric used toevaluate an entity. Any of these changes based on a new or revisedassessment may cause a remediation service and/or a new or additionalassessment service, to be implemented. Trends, averages and/or changesto an assessment or evaluation may impact policy criteria of a cybersecurity policy, as discussed herein.

Various embodiments of the present technology can be practiced with alocal computer system, and/or a cloud-based system. FIG. 5 is a highlevel schematic diagram of a computing architecture (hereinafterarchitecture 500) of the present technology. The architecture 500comprises a system 505, which in some embodiments comprises a server orcloud-based computing device configured specifically to perform thediversity analyses described herein. That is, the system 505 is aparticular purpose computing device that is specifically designed andprogrammed (e.g., configured or adapted) to perform any of the methodsdescribed herein. The system 505 can be coupled with entity device 530using a network 520.

In one embodiment, the system 505 comprises a processor 510 and memory515 for storing instructions. The memory 515 can include arecommendation module 540. As used herein, the terms “module” may alsorefer to any of an application-specific integrated circuit (“ASIC”), anelectronic circuit, a processor (shared, dedicated, or group) thatexecutes one or more software or firmware programs, a combinationallogic circuit, and/or other suitable components that provide thedescribed functionality.

The system 505 may gather variables for an entity by querying the entityfor information, scraping available online sources such as websites,corporate filings, news sources, other public record databases, andother resources. Additionally, data may be gathered from the entity'snetwork using devices already present there or by placing a new deviceon the entity's network to gather more data. The data collecting devicemay be a server, router, firewall, switch, or repeater, or may be asoftware agent or routine that monitors traffic and/or performs packetinspection. The data collecting device may be on the company's networkand/or its periphery, and may collect and/or analyze the data, whilealso transmitting it to system 505. In this manner, additional,proprietary data may be gleaned from a particular entity's network. Thevariables or a subset of the variables can be compared. The comparisoncan be for all or only a subset of all entities. The subset of variablescan be selected by the end user, as well as the entities analyzed.

In some embodiments, the system 505 provides interfaces or adapters505A-N that allow various resources to communicatively couple with thesystem 505. As an example, the system 505 can use an application programinterface (API) or other communication interface. FIG. 5 illustratesexample resources that can couple with the system 505. The system 505can interrogate, for example, various databases such as corporatefilings, news sources, and other public record databases. In anotherexample, cloud services such as cloud storage and cloud computingenvironments. In general, a cloud-based computing environment is aresource that typically combines the computational power of a largegrouping of processors and/or that combines the storage capacity of alarge grouping of computer memories or storage devices. For example,systems that provide a cloud resource may be utilized exclusively bytheir owners; or such systems may be accessible to outside users whodeploy applications within the computing infrastructure to obtain thebenefit of large computational or storage resources. The cloud may beformed, for example, by a network of servers with each server (or atleast a plurality thereof) providing processor and/or storage resources.These servers may manage workloads provided by multiple users (e.g.,cloud resource customers or other users). Typically, each user may placeworkload demands upon the cloud that vary in real-time, sometimesdramatically. The nature and extent of these variations typically dependon the type of business associated with the user.

The system 505 may also couple with the Internet as well as data feedssuch as RSS feeds or social networks. Email behaviors can also beidentified by interrogating email servers or email repositories.

In some embodiments, the system 505 can use vulnerability assessmentsgenerated by the entity or a third party, such as a cyber-security firm.

In contrast with a vulnerability assessment, which is more technical innature, the present technology can also consider non-technical orsemi-technical aspects of an entity and how these elements impact thecyber vulnerability of the entity. For example, non-technical elementsinclude, but are not limited to, company size, revenue, companylocation, company industry sector, as well as other elements which aredescribed herein. The present technology provides benefits above andbeyond a typical vulnerability assessment, providing users with a robustand comprehensive view of a company's (or multiple companies') overallcyber security risk.

In some embodiments, the system 505 can obtain sophisticationinformation about entities from the following non-limiting list ofsources or resources: (a) Framework; (b) Hosting/infrastructure; (c)Account management; (d) Authentication; (e) Authorization; (f) Scanning;(g) System vulnerability; (h) Ad/Partner integration; (i)Files/Directories/Links; and (j) Patching.

In some embodiments, the system 505 can obtain sophisticationinformation about entities from the following non-limiting list ofsources or resources: (a) Customer Reviews; (b) Employee reviews; (c)Traffic statistics; (d) Business events/news; (e) Corporate connections;(f) Business type; (g) Customer data; (h) Brand/Revenue; (i) Employeeprofiles; (j) Social Media/Blogs; (k) Industry/Products; (l) Data Types;and (m) Company/Subsidiary connections.

For purposes of context, facets or features relating the motivationregarding a security failure (e.g., motivation of some actor, such as ahacker, to attack an entity, to expose sensitive information, to name afew) as well as the sophistication of the entity in preventing ordealing with a cyber security event will be referred to herein as anelement. Thus, there can be a plurality of types of sophisticationelements and a plurality of types of motivation elements. The actor maybe a hacker, employee, another entity, to name a few.

Examples of motivation elements include: visibility; value; hackersentiment; employee sentiment; company sentiment; customer sentiment,and combinations thereof—just to name a few. Each of these motivationelements may be further subcategorized as follows. Visibility mayinclude information and/or derived measures related to the traffic,usage, and activity related to an entity, including but not limited tothe in-links; pageviews; duration; traffic; links; page rank; marketvalue; daily (stock) trade volume; exporting/importing; and combinationsthereof—just to name a few. Value includes: revenue; net income; totalassets; employees; and combinations thereof—just to name a few. Hackersentiment includes: emails; credit cards; foreign languages; etc., whichcan be gathered from hacker forums and/or discussion groups, chat rooms,dark web, or dark net forums, such as the Tor Network, Internet RelayChat (IRC), and combinations thereof—just to name a few. Employeesentiment includes: career opportunities; work/life balance;compensation; and combinations thereof—just to name a few. Companysentiment includes: senior leadership ratings; overall company ratings;recommendations; etc. Customer sentiment includes: product ratings;service ratings, and combinations thereof—just to name a few.

The present technology determines a level of sophistication of theentity. Sophistication may be considered a measure of People, Process,and Technology. People indicates how security-aware the entities'employees, principals and/or members are. In particular, do the peopleassociated with the entity understand the risks, are they competent insecurity, and combinations thereof. Process indicates whether proceduresand/or policies have clear and enforceable terms, and clearly indicatewhat to do in case of various events, including attacks. Process alsoindicates whether training is provided to employees, third partycontractors and/or service providers, indicates their level ofexpertise, and combinations thereof.

Examples of sophistication elements include: hosting infrastructure;topology; vulnerability scanning; people; and combinations thereof—justto name a few. Hosting infrastructure includes; content distributionnetworks; shared hosting; cloud providers; etc. Topology includes:accessibility points; page layout; content on site; etc. Vulnerabilityscanning includes: CVEs (common vulnerabilities and exposures);patching; updating; default passwords; etc. People includes: chiefinformation security officer (CISO); security team; skills; jobpostings; etc. In this manner, sophistication encompasses more than justvulnerability, and additionally includes people and processes that mayimpact a defensive posture of an entity.

Determining these variables may be a data gathering operation, which maybe based on public information or a company's own data networks, asdiscussed herein. A cyber risk assessment, for instance a two by two (orhigher order) graph, may be output, along with a composite score, a peerrank, an estimated commercial impact, and recommendations to decreasethe cyber risk. These may all be output for each company assessed. Allof these elements may be updated over time and in response toimplementation of recommendations, thus, transforming the original datavia the use of a particular computer.

In some embodiments, the system 505 is configured to evaluate each datapoint with respect to history, lineage, provenance (e.g., origin),source, time, entities and other details. The system 505 can thencleanse and standardize the data points. Examples of cleansing andstandardizing using data normalization are described in greater detailbelow.

In some embodiments, the system 505 can use a canonical representationof the data points. As mentioned above, the system 505 can trackentities and their attributes/elements over time. The system 505 is alsoconfigured to process rollups (e.g., summarizing the data along adimension), aggregations, transforms, reductions, normalizations,deltas, as well as other types of data transformation or conversionprocesses that can also be used to convert themotivation/sophistication/combination elements into scores.

The system 505 then generates module-ready data for use with matrices ofelements (motivation/sophistication) for one or more entities. In someembodiments, the system 505 then executes one or more models to generatescores, results, recommendations, delta values (changes in scores overtime), as well as historical tracking of scores.

In some embodiments, the system 505 comprises a scoring and plottingmodule 535 that is generally configured to calculate sophisticationscores, motivation scores, and combination scores; apply weighting tosophistication and/or motivation elements in various calculations;compare scores to threshold values; benchmark various scores over time;as well as other features described herein.

In a second set of functions, the scoring and plotting module 535 cancreate visual representations such as the graphs illustrated in FIGS.6-12.

In one embodiment, the scoring and plotting module 535 is configured tocalculate various scores for an entity. In another embodiment thescoring and plotting module 535 can calculate various scores for aplurality of entities. Again, these various scores can be calculatedover time and utilized for benchmarking cyber security performance foran entity, or a group of entities that possess a particular attribute incommon. For example, the scoring and plotting module 535 can calculatescores for groups of entities in an industry group, a geographicallocation, a company size, a technology sector, and so forth.

In an example calculation, the scoring and plotting module 535 isconfigured to calculate a motivation score for one or more entities. Thescoring and plotting module 535 obtains motivation elements collectedfrom the various resources and converts this information into amathematical representation. In one embodiment, a motivation element ofpageviews can be mathematically represented by comparing the pageviewsof the entity to a set of thresholds. For context, the pageviews couldbe a pageview of a particular webpage or set of webpages. To be sure,the higher profile and more visited a website is, the more likely thatit will be attractive to a hacker, especially if other motivationfactors are present such as the entity being involved in commercialactivities, just for example.

For purposes of obtaining a coherent scoring scheme, the scoring andplotting module 535 may normalize various elements to obtainmathematical values that are usable in an algorithm for scoringmotivation or sophistication. By way of example, each of the set ofthresholds is associated with a mathematical value. If the entity haspageviews in excess of 10,000 unique users in one day, the entity isgiven a score of five. If the entity has pageviews in excess of 100,000unique users in one day, the entity is given a score of ten. If theentity has pageviews in excess of 200,000 unique users in one day, theentity is given a score of fifteen. Again, these are merely examples ofpossible ways to convert pageviews into a mathematical representationthat can be combined with other mathematical representations of othermotivation elements in order to create an overall motivation score.

In other examples, an employee sentiment can be representedmathematically as a percentage of positive versus negative comments fromemployees. In another example, negative employee behaviors, actions, orstatements can be counted over time and compared to thresholds (in amethod similar to that above with respect to pageviews).

Each of the motivation elements (if necessary) is converted into amathematical representation. The ultimate motivation score can becalculated by taking a sum of each mathematical representation ofmotivation elements. In some embodiments, the motivation score can be arepresentation of one or a combination of many motivation elements.

In some embodiments, the system 505 can be configured to weight one ormore of the elements in a score calculation. For example, if it isdetermined that certain elements are more likely to increase thelikelihood of a security failure (e.g., a cyber attack, a privacyincident), these elements can be assigned a weight. In an example, theweight is applied by multiplying a mathematical representation of anelement by a coefficient or factor. If an element value for pageviews isfive, a weighting could include multiplying this number by a coefficientof 0.5, which reduces the impact of that value on the overall score.Increases in element values can also be achieved.

While the above examples reference motivation elements, the scoring andplotting module 535 is also configured to process sophisticationelements to obtain sophistication scores. The exact details forconverting sophistication/motivation elements into mathematicalrepresentations will vary according to the type of information includedin the elements. To be sure, some types of elements such as pageviewsand revenue are inherently more mathematical in their quantities, whileother elements are more non-mathematical in nature such as employee orcustomer sentiment. For non-mathematical elements, users can developsuitable schemes or algorithms for converting or quantifying theseelements into mathematical form.

According to some embodiments, the scoring and plotting module 535 candetermine various facets of an entity or group of entities by comparingthe motivation, sophistication, and/or combined scores of theseentities. Answers to pertinent questions can be deduced or inferred fromthe comparison.

For example, in one embodiment, the scoring and plotting module 535 isconfigured to determine a position of an entity within an aggregate riskscore of a portfolio of entities. Thus, the scoring and plotting module535 has been used to calculate an aggregate risk score (motivation,sophistication, and/or combined) for numerous entities. In oneembodiment, the scoring and plotting module 535 selects a plurality ofmotivation elements and analyzes these elements for each of a portfolio(plurality) of entities using the above examples as a guide forcalculating motivation scores. In some embodiments, the same motivationelements are used for each entity.

The scoring and plotting module 535 can then determine where the entitylies within the group of scores. For example, out of 30 entities, asubject entity places 25th out of thirty.

The scoring and plotting module 535 can also be utilized to generategraphs and GUIs that display various scores in graphical format(s). Forexample, in FIG. 6, a graph with two axes is illustrated. The graph 600comprises a vertical axis that is representative of motivation elements,and the horizontal axis is representative of sophistication elements.Indeed, this graph can be used to display information about a singleentity or a plurality of entities.

In one embodiment, the motivation axis is delineated or stratified basedon the type of content. Less important types of secure information arelocated towards the bottom of the axis, whereas more important types ofinformation are located at the top part of the axis. In this embodiment,the lower part of the motivation axis references payment cards (e.g.,credit cards) and other types of general consumer information. Abovethat is online crime such as phishing, malware, and other maliciousbehavior. Above online crime is IP theft and industrial espionage. Atthe top of the motivation axis are state secrets. To be sure, othercategories of information types will lie somewhere along this axis, ifnot specifically mentioned. Furthermore, the axis can be defined byother types of information points. For example, an entity can structuretheir motivation axis to include information that they deal with,structured from least important to most important.

In the sophistication axis, which is the horizontal axis, hackerprofiles are listed from left to right on the axis from a lowest impactactor type to a highest impact actor type. For example, actor types caninclude casual hackers, professional hackers, organized crime, and stateactors. Each of these actor types has a different threat levelassociated therewith. The sophistication axis represents the strength orthreat level that it takes to successfully hack the subjectentity/entities.

FIG. 7 is an example graphical user interface (GUI) that comprisesscatter plot illustrating an entity's motivation and sophisticationrelative to cyber risk. The scatter plot 700 comprises a verticalmotivation axis and a horizontal sophistication axis. Each of the pointsplotted on the scatter plot 700 represent an entity. Again, theseentities can be analyzed together because they are a part of an entitygroup (e.g., industry group, same geographical location, same companysize, etc.).

FIG. 8 is an example graphical user interface (GUI) that comprises a bargraph illustrating the plotting of a plurality of entities based ontheir combination scores. The bar graph 800 comprises a vertical axisthat represents a number of companies and a horizontal axis thatrepresents combination scores for a set of entities. For example, mostentities in the group have combination scores (sophistication andmotivation) that fall within a score range of 51-60. Other groups ofentities fall within other score ranges.

To be sure the system 505 can cause an elemental analysis of thesesimilar scoring groups to identify what elements are shared between theentities, what elements are different, and so forth. Thus, the graphingof entities based on scores aids the system 505 in identifying groups ofentities that require attention. For example, the entities in the scorerange of 31-40 are severely underperforming.

FIG. 9 is an example graphical user interface (GUI) that comprises a bargraph illustrating the plotting of a plurality of entities based ontheir sophistication scores. The bar graph 900 comprises a vertical axisthat represents a number of companies and a horizontal axis thatrepresents sophistication scores for a set of entities.

FIG. 10 is an example graphical user interface (GUI) that comprises abar graph illustrating the plotting of a plurality of entities based ontheir motivation scores. The bar graph 1000 comprises a vertical axisthat represents a number of companies and a horizontal axis thatrepresents motivation scores for a set of entities.

By comparing these graphs illustrated in FIGS. 8-10, underperformance insophistication and/or motivation can be quickly and easily determined,at least on a high level. Again, a more granular element analysis can beconducted when groups with underperforming sophistication/motivationscores are identified.

FIG. 11 is an example graphical user interface (GUI) that comprises ascatter plot that represents a plurality of entities plotted accordingto their combination score. The scatter plot 1100 includes a pluralityof data points that each represents an entity. The scatter plot 1100comprises a vertical axis that represents motivation and a horizontalaxis that represents sophistication scores for a set of entities. Thehigher risk area on the plot is where the motivation to hack is high andthe sophistication of the entity is low.

The system 505 can create a line 1105 of acceptablemotivation/sophistication scores. Companies falling below this line 1105have a suitable cyber risk profile, whereas companies above the linehave an unsuitable cyber risk profile. These companies can be identifiedand analyzed in order to suggest recommendations for improving theircyber risk.

FIG. 12 is an example graphical user interface (GUI) 1200 that comprisesa scatter plot that represents a plurality of entities plotted accordingto their combination score, as well as additional graphicalrepresentations for an entity and a list of recommendations based on theplotting. The scatter plot includes a line 1205 of acceptablemotivation/sophistication scores.

The plot is similar to that of FIG. 11, with the addition of twographical representations. For example, a linear slide 1220 displays theposition of an entity within a peer group of entities. This samerelationship position is illustrated in a gauge graph 1210.

In response to making a cyber risk assessment, the recommendation module540 can be executed to provide the end user (or entity) with some typeof actionable feedback. For example, the recommendation module 540 canprovide the end user one or more actions to the end user based on thediversity score and the clusters of similar variables. This is describedin further detail above. These one or more actions potentially decreasethe cyber risk of the entity. In one example, the recommendation module540 can automatically identify variables, which if changed, would affectthe cyber risk assessment. In further exemplary embodiments, entitiesmay agree to automatic implementation of recommendations in exchange forlower policy premiums.

As best illustrated in FIG. 12, a set of recommendations 1215 isprovided along with the graphical analysis generated for the entity.Again, these recommendations are based on the system 505 havingknowledge of the motivation elements, sophistication elements, as wellas the scores calculated not only for the entity, but other entities (insome embodiments).

Exemplary methods and systems according to the present technology mayalso provide benchmarking over time. In this manner, the system 505 cantrack, for a company or group or entities, cyber risk over a selectabletime period, for example days, weeks, months, and/or years. Thisbenchmarking may be against a dynamic or static evaluation of the peergroup, for instance, an entity's past and present cyber risk trackedagainst a static past peer group, static present peer group, and/ordynamic peer group. The present technology provides information relatedto the updated information (the new motivation score, the newsophistication score, the new composite score, etc.), includingdifferences (the amount of the change made in one or more updates,namely the delta), and trends (patterns over many time steps).

FIG. 13 is a flowchart of an example method 1300 of the presenttechnology. The method 1300 includes the system 505 assessing 1305 riskof a cyber security failure in a computer network of an entity, using acomputer agent configured to collect information from at least publiclyaccessible Internet elements. The cyber risk includes a security failure(e.g., a cyber attack, a privacy incident) of the entity.

The system 505 may query the entity for information, scrape availableonline sources such as websites, corporate filings, news sources, otherpublic record databases, and other resources. Additionally, data may begathered from the entity's network using devices already present thereor by placing a new data collecting device on the entity's network togather more data. The data collecting device may be on the company'snetwork and/or its periphery, and may collect and/or analyze the data,while also transmitting it to the system 505. In this example,additional, proprietary data may be gleaned from a particular entity'snetwork.

The exemplary method 1300 also includes the system 505 automaticallydetermining 1310, based on the assessed risk, a change or a setting toat least one element of policy criteria of a cyber security policy. Insome embodiments, the at least one element of policy criteria involves aterm or condition of a cyber policy. In one embodiment, the cybersecurity policy includes a cyber security insurance policy.

Next, in this example, the method 1300 includes the system 505automatically recommending 1315, based on the assessed risk, computernetwork changes to reduce the assessed risk

Next, in this example, the method 1300 includes the system 505 providing1320 one or more recommended computer network changes to reduce theassessed risk, enactment by the entity of at least one of the one ormore of the recommended computer network changes to reduce the assessedrisk to the entity.

The exemplary method 1300 also includes the system 505 determining 1325that the entity has enacted at least a portion of the recommendedcomputer network changes, and in response, automatically reassessing thecyber security risk of a cyber security failure in the computer networkof the entity based on the enacted recommended computer network changes.

Next, the exemplary method 1300 includes the system 505 dynamicallyre-determining 1330, based on the reassessed risk of a cyber securityfailure in the computer network of the entity, the change or the settingto the at least one element of policy criteria of the cyber securitypolicy.

FIG. 14 is a flowchart of an example method 1400. The method 1400includes the system 505, based on the assessing of risk of the cybersecurity failure in the computer network of the entity, plotting 1405one or more features of the entity and other members of a peer group ofthe entity, the plotting being configured to visually illustrate therisk of a cyber security failure in the computer network of the entity;and the automatically recommending of computer network changes beingbased on the plotting.

Next, the method 1400 includes the system 505, in response to thedetermining that the entity has enacted at least a portion of therecommended computer network changes, initiating 1410 the change or thesetting to the at least one element of policy criteria of the cybersecurity policy.

Next, the method 1400 for the assessing of risk of the cyber securityfailure in the computer network of the entity includes the system 505,using 1415 a plurality of sophistication elements for the entity, asophistication for the entity with respect to preventing the cybersecurity failure, the sophistication being one of features of theentity. Again, the sophistication relates to people, processes, andtechnology. The sophistication analysis as a whole attempts to quantifyhow strong a threat actor would be required to execute a successfulsecurity failure of the entity.

Next, the method 1400 for the assessing of risk of the cyber securityfailure in the computer network of the entity includes the system 505assessing, using a plurality of motivation elements regarding theentity, a motivation of an actor to initiate the cyber security failure,the motivation being one of a plurality of features of the entity. Insome embodiments motivation is a motivation of an actor (e.g., a hacker)to initiate a cyber security failure.

FIG. 15 is a flowchart of yet another example method 1500 for modifyinga policy based on a cyber risk analysis. The method 1500 includes thesystem 505 assessing 1505, using a plurality of sophistication elementsfor the entity, a sophistication for the entity with respect topreventing the cyber security failure, the sophistication being one of aplurality of features of the entity; and assessing, using a plurality ofmotivation elements regarding the entity, a motivation of an actor toinitiate the cyber security failure, the motivation being another one ofthe features of the entity. Again, the sophistication relates to people,processes, and technology. The sophistication analysis as a wholeattempts to quantify how strong a threat actor would be required tocause a successful cyber failure.

Next, the method 1500 in various embodiments includes the system 505calculating 1510 a composite score from a motivation score and asophistication score, the motivation score representing the plurality ofmotivation elements, the sophistication score representing the pluralityof sophistication elements.

To be sure, steps 1505 and 1510 include the collection of motivation andsophistication elements, converting these elements into mathematicalrepresentations (if needed), and processing these elements into scoresusing relevant algorithms.

In some embodiments, the method 1500 includes the system 505 creating1515 an aggregate risk score of a portfolio of entities based on aplurality of motivation scores including the motivation score and aplurality of sophistication scores including the sophistication score;and benchmarking over time at least one of the sophistication score, themotivation score, the composite score, and the aggregate risk score.

Next, the method 1500 in some embodiments includes the system 505identifying 1520 clusters of sophistication elements or motivationelements shared between two or more entities of the portfolio ofentities, the clusters of sophistication elements or motivation elementsbeing associated with an increase in risk of a cyber security failure inthe computer network of the entity.

Next, the method 1500 in some instances includes the system 505identifying 1525 additional sophistication elements or motivationelements for at least one of the two or more entities of the portfolioof entities that are not shared with the portfolio of entities, theadditional sophistication elements or motivation elements beingassociated with another increase in the risk of a cyber security failurein the computer network of the entity; and generating recommendationsfor the at least one of the two or more entities of the portfolio ofentities that will cause a decrease in the risk of a cyber securityfailure in the computer network of the entity.

According to some embodiments, the system 505 can be programmed withpolicy parameters. The system 505 can generate recommendations for theinsurer based on the motivation and sophistication analysis of theentity. In some instances, the recommendation could be to deny a policyor terminate a policy if the entity has motivation or sophisticationelements that are defined by the policy as being unacceptable oruninsurable.

According to some embodiments, the present disclosure can includesystems and methods that utilize the vast array of information gatheredby computer agents and provided by entities to discover circumstantialor indirect evidence about an entity that may affect the entity'sassessed risk.

For context, data collected by the systems and methods of the presentdisclosure, such as system 505 (FIG. 5), may obtain a large amount ofdata (e.g., big data collection) that can include many types of datathat directly reference, partially reference, or indirectly reference anentity. A direct reference is a reference to an entity that isconfirmable, meaning that it is virtually certain that the entity isdiscussed or referenced by a particular data. An example would be anewspaper article.

A partial reference could include any data that mentions the entity, butreferences the entity in an incomplete manner. For example, an onlinereview where the entity has been named, but the author did not spell theentity's name in an exact manner. Thus, the partial reference is asimple process to confirm whether the data is truly referencing theentity.

Data using an indirect reference may be more difficult to analyze. Forexample, a social media user may write a lengthy complaint about aparticular entity, but the author may not want to name the entitydirectly. Through big data collection processes and circumstantial orindirect evidence analysis using cross referencing of the big datacontent, the system 505 can confirm that an entity is being referencedby the author. By way of example, suppose that the author is an employeeof the entity and is writing a whistle blowing article about workplacebehaviors of the entity. Using collected data, the system can crossreference the author's name and workplace with entity information storedin the system to determine or infer that the employee is describingtheir workplace.

As additional information and data are collected and stored by thesystem 505, the ability of the system 505 to cross reference dataincreases, which also increases the likelihood that circumstantial orindirect linkages will be determined and found useful for the diversityanalyses described herein.

To be sure, these circumstantial evidentiary reviews may be utilized inrisk assessment. If the circumstantial or indirect evidence is positive,the cyber risk may be decreased. Conversely, if the circumstantial orindirect evidence is negative, the cyber risk may be increased.

Also, in some embodiments, a relative strength of the cross referencedcircumstantial or indirect evidence can be determine and utilized. Forexample, if the system cannot determine with certainty that the entityis implicated in the circumstantial or indirect evidence, or if thecircumstantial or indirect evidence is negative or positive in nature,the system 505 can choose to ignore the circumstantial or indirectevidence within the diversity analysis or flag the entity for additionalreview.

FIG. 16 is a flowchart of an example method 1600 of the presentdisclosure. The method 1600 includes the system 505 assessing 1605 riskof a cyber security failure in a computer network of an entity, using acomputer agent configured to collect information from at least publiclyaccessible Internet elements. The cyber risk includes a security failure(e.g., a cyber-attack, a privacy incident) of the entity.

The system 505 may query the entity for information, scrape availableonline sources such as websites, corporate filings, news sources, otherpublic record databases, and other resources. Additionally, data may begathered from the entity's network using devices already present thereor by placing a new data collecting device on the entity's network togather more data. The data collecting device may be on the company'snetwork and/or its periphery, and may collect and/or analyze the data,while also transmitting it to system 505. In this example, additional,proprietary data may be gleaned from a particular entity's network.

In various embodiments, a component of this review process includes acircumstantial or indirect data or information review for the entity.Thus, the method 1600 includes a sub-process that involves a step 1605-Aof evaluating the collected information to obtain circumstantial orindirect information that is indicative of the entity. In one example,the process comprises scanning the social media accounts of employees ofthe entity, as well as any other online resources linked to theemployees. Another example includes searching trade journals or legalproceedings. In another example, the process utilizes data gatheredduring routine use of the system 505 in gathering data for direct cyberrisk assessments and extracting circumstantial or indirect informationtherefrom.

Next, the sub-process comprises a step 1605-B of cross referencing datain the collected information to confirm or infer that the entity isreferenced in the circumstantial or indirect information that isindicative of the entity being referenced in the circumstantial orindirect information.

The sub-process can also include a step 1605-C of at least one ofincreasing or decreasing the assessed risk if the circumstantial orindirect information is negative or positive.

Also, as mentioned above strength of the circumstantial or indirectanalysis can be utilized in some embodiments. For example, if thecircumstantial or indirect analysis does not conclude that the entityreferenced in the circumstantial or indirect data is the entity of thesubject cyber risk analysis, but that it is highly likely that theentity is correct, the system 505 can weigh the circumstantial orindirect analysis aspect of the risk analysis by 80 percent. If theentity is later determined to be correct through additional datagathering, a weight of the circumstantial or indirect analysis can beincreased to 100 percent.

Thus, in some embodiments, the circumstantial or indirect analysis canbe periodically executed to refine the cyber risk analyses of thepresent disclosure. In one embodiment, the circumstantial or indirectanalysis is executed during re-assessment and/or re-determining periodsdescribed herein. In other embodiments, the circumstantial or indirectanalysis is executed and/or re-executed at other intervals.

By way of example, if the risk analysis calculates a circumstantialscore of seven for the entity (where scores fall in a range of zero to10), and the system 505 has a certainty level of 80 percent, the scorewould be reduced to 5.6.

The exemplary method 1600 also includes the system 505 automaticallydetermining 1610, based on the assessed risk, a change or a setting toat least one element of policy criteria of a cyber security policy. Insome embodiments, the one or more elements of policy criteria involve aterm or condition of a cyber policy. In one embodiment, the cyber policyincludes a cyber security insurance policy.

Next, in this example, the method 1600 includes the system 505automatically recommending 1615, based on the assessed risk, computernetwork changes to reduce the assessed risk.

Next, in some instances, the method 1600 includes the system 505providing 1620 one or more recommended computer network changes toreduce the assessed risk, enactment by the entity of at least one of theone or more of the recommended computer network changes to reduce theassessed risk to the entity.

The exemplary method 1600 also includes the system 505 determining 1625determining that the entity has enacted at least a portion of therecommended computer network changes, and in response, automaticallyreassessing the cyber security risk of a cyber security failure in thecomputer network of the entity based on the enacted recommended computernetwork changes.

Next, the exemplary method 1600 includes the system 505 dynamicallyre-determining 1630, based on the reassessed risk, the change or thesetting to the at least one element of policy criteria of the cybersecurity policy.

FIG. 17 is a diagrammatic representation of an example machine in theform of a computer system 1, within which a set of instructions forcausing the machine to perform any one or more of the methodologiesdiscussed herein may be executed. In various example embodiments, themachine operates as a standalone device or may be connected (e.g.,networked) to other machines. In a networked deployment, the machine mayoperate in the capacity of a server or a client machine in aserver-client network environment, or as a peer machine in apeer-to-peer (or distributed) network environment. The machine may be apersonal computer (PC), a tablet PC, a set-top box (STB), a personaldigital assistant (PDA), a cellular telephone, a portable music player(e.g., a portable hard drive audio device such as an Moving PictureExperts Group Audio Layer 3 (MP3) player), a web appliance, a networkrouter, switch or bridge, or any machine capable of executing a set ofinstructions (sequential or otherwise) that specify actions to be takenby that machine. Further, while only a single machine is illustrated,the term “machine” shall also be taken to include any collection ofmachines that individually or jointly execute a set (or multiple sets)of instructions to perform any one or more of the methodologiesdiscussed herein.

The example computer system 1 includes a processor or multipleprocessor(s) 5 (e.g., a central processing unit (CPU), a graphicsprocessing unit (GPU), or both), and a main memory 10 and static memory15, which communicate with each other via a bus 20. The computer system1 may further include a video display 35 (e.g., a liquid crystal display(LCD)). The computer system 1 may also include an alpha-numeric inputdevice(s) 30 (e.g., a keyboard), a cursor control device (e.g., amouse), a voice recognition or biometric verification unit (not shown),a drive unit 37 (also referred to as disk drive unit), a signalgeneration device 40 (e.g., a speaker), and a network interface device45. The computer system 1 may further include a data encryption module(not shown) to encrypt data.

The disk drive unit 37 includes a computer or machine-readable medium 50on which is stored one or more sets of instructions and data structures(e.g., instructions 55) embodying or utilizing any one or more of themethodologies or functions described herein. The instructions 55 mayalso reside, completely or at least partially, within the main memory 10and/or within the processor(s) 5 during execution thereof by thecomputer system 1. The main memory 10 and the processor(s) 5 may alsoconstitute machine-readable media.

The instructions 55 may further be transmitted or received over anetwork (e.g., network 105B or network 520, see FIG. 1 and FIG. 5,respectively) via the network interface device 45 utilizing any one of anumber of well-known transfer protocols (e.g., Hyper Text TransferProtocol (HTTP)). While the machine-readable medium 50 is shown in anexample embodiment to be a single medium, the term “computer-readablemedium” should be taken to include a single medium or multiple media(e.g., a centralized or distributed database and/or associated cachesand servers) that store the one or more sets of instructions. The term“computer-readable medium” shall also be taken to include any mediumthat is capable of storing, encoding, or carrying a set of instructionsfor execution by the machine and that causes the machine to perform anyone or more of the methodologies of the present application, or that iscapable of storing, encoding, or carrying data structures utilized by orassociated with such a set of instructions. The term “computer-readablemedium” shall accordingly be taken to include, but not be limited to,solid-state memories, optical and magnetic media, and carrier wavesignals. Such media may also include, without limitation, hard disks,floppy disks, flash memory cards, digital video disks, random accessmemory (RAM), read only memory (ROM), and the like. The exampleembodiments described herein may be implemented in an operatingenvironment comprising software installed on a computer, in hardware, orin a combination of software and hardware.

One skilled in the art will recognize that the Internet service may beconfigured to provide Internet access to one or more computing devicesthat are coupled to the Internet service, and that the computing devicesmay include one or more processors, buses, memory devices, displaydevices, input/output devices, and the like. Furthermore, those skilledin the art may appreciate that the Internet service may be coupled toone or more databases, repositories, servers, and the like, which may beutilized in order to implement any of the embodiments of the disclosureas described herein.

The corresponding structures, materials, acts, and equivalents of allmeans or step plus function elements in the claims below are intended toinclude any structure, material, or act for performing the function incombination with other claimed elements as specifically claimed. Thedescription of the present technology has been presented for purposes ofillustration and description, but is not intended to be exhaustive orlimited to the present technology in the form disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of the presenttechnology. Exemplary embodiments were chosen and described in order tobest explain the principles of the present technology and its practicalapplication, and to enable others of ordinary skill in the art tounderstand the present technology for various embodiments with variousmodifications as are suited to the particular use contemplated.

Aspects of the present technology are described above with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products according to embodiments of thepresent technology. It will be understood that each block of theflowchart illustrations and/or block diagrams, and combinations ofblocks in the flowchart illustrations and/or block diagrams, can beimplemented by computer program instructions. These computer programinstructions may be provided to a processor of a general purposecomputer, special purpose computer, or other programmable dataprocessing apparatus to produce a machine, such that the instructions,which execute via the processor of the computer or other programmabledata processing apparatus, create means for implementing thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

These computer program instructions may also be stored in a computerreadable medium that can direct a computer, other programmable dataprocessing apparatus, or other devices to function in a particularmanner, such that the instructions stored in the computer readablemedium produce an article of manufacture including instructions whichimplement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The computer program instructions may also be loaded onto a computer,other programmable data processing apparatus, or other devices to causea series of operational steps to be performed on the computer, otherprogrammable apparatus or other devices to produce a computerimplemented process such that the instructions which execute on thecomputer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods and computer program products according to variousembodiments of the present technology. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat, in some alternative implementations, the functions noted in theblock may occur out of the order noted in the figures. For example, twoblocks shown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in the reverseorder, depending upon the functionality involved. It will also be notedthat each block of the block diagrams and/or flowchart illustration, andcombinations of blocks in the block diagrams and/or flowchartillustration, can be implemented by special purpose hardware-basedsystems that perform the specified functions or acts, or combinations ofspecial purpose hardware and computer instructions.

In the following description, for purposes of explanation and notlimitation, specific details are set forth, such as particularembodiments, procedures, techniques, etc. in order to provide a thoroughunderstanding of the present invention. However, it will be apparent toone skilled in the art that the present invention may be practiced inother embodiments that depart from these specific details.

Reference throughout this specification to “one embodiment” or “anembodiment” means that a particular feature, structure, orcharacteristic described in connection with the embodiment is includedin at least one embodiment of the present invention. Thus, theappearances of the phrases “in one embodiment” or “in an embodiment” or“according to one embodiment” (or other phrases having similar import)at various places throughout this specification are not necessarily allreferring to the same embodiment. Furthermore, the particular features,structures, or characteristics may be combined in any suitable manner inone or more embodiments. Furthermore, depending on the context ofdiscussion herein, a singular term may include its plural forms and aplural term may include its singular form. Similarly, a hyphenated term(e.g., “on-demand”) may be occasionally interchangeably used with itsnon-hyphenated version (e.g., “on demand”), a capitalized entry (e.g.,“Software”) may be interchangeably used with its non-capitalized version(e.g., “software”), a plural term may be indicated with or without anapostrophe (e.g., PE's or PEs), and an italicized term (e.g., “N+1”) maybe interchangeably used with its non-italicized version (e.g., “N+1”).Such occasional interchangeable uses shall not be consideredinconsistent with each other.

Also, some embodiments may be described in terms of “means for”performing a task or set of tasks. It will be understood that a “meansfor” may be expressed herein in terms of a structure, such as aprocessor, a memory, an I/O device such as a camera, or combinationsthereof. Alternatively, the “means for” may include an algorithm that isdescriptive of a function or method step, while in yet other embodimentsthe “means for” is expressed in terms of a mathematical formula, prose,or as a flow chart or signal diagram.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the invention. Asused herein, the singular forms “a”, an and the are intended to includethe plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises”and/or “comprising,” when used in this specification, specify thepresence of stated features, integers, steps, operations, elements,and/or components, but do not preclude the presence or addition of oneor more other features, integers, steps, operations, elements,components, and/or groups thereof.

It is noted at the outset that the terms “coupled,” “connected”,“connecting,” “electrically connected,” etc., are used interchangeablyherein to generally refer to the condition of beingelectrically/electronically connected. Similarly, a first entity isconsidered to be in “communication” with a second entity (or entities)when the first entity electrically sends and/or receives (whetherthrough wireline or wireless means) information signals (whethercontaining data information or non-data/control information) to thesecond entity regardless of the type (analog or digital) of thosesignals. It is further noted that various figures (including componentdiagrams) shown and discussed herein are for illustrative purpose only,and are not drawn to scale.

While specific embodiments of, and examples for, the system aredescribed above for illustrative purposes, various equivalentmodifications are possible within the scope of the system, as thoseskilled in the relevant art will recognize. For example, while processesor steps are presented in a given order, alternative embodiments mayperform routines having steps in a different order, and some processesor steps may be deleted, moved, added, subdivided, combined, and/ormodified to provide alternative or sub-combinations. Each of theseprocesses or steps may be implemented in a variety of different ways.Also, while processes or steps are at times shown as being performed inseries, these processes or steps may instead be performed in parallel,or may be performed at different times.

While various embodiments have been described above, it should beunderstood that they have been presented by way of example only, and notlimitation. The descriptions are not intended to limit the scope of theinvention to the particular forms set forth herein. To the contrary, thepresent descriptions are intended to cover such alternatives,modifications, and equivalents as may be included within the spirit andscope of the invention as defined by the appended claims and otherwiseappreciated by one of ordinary skill in the art. Thus, the breadth andscope of a preferred embodiment should not be limited by any of theabove-described exemplary embodiments.

What is claimed is:
 1. A method, comprising: assessing risk of a cybersecurity failure in a computer network of an entity, using a computeragent configured to collect information from at least publiclyaccessible Internet elements, wherein the assessing of risk comprises:evaluating the collected information to obtain circumstantial orindirect information that is indicative of the entity; cross referencingdata in the collected information to confirm or infer that the entity isreferenced in the circumstantial or indirect information that isindicative of the entity being referenced in the circumstantial orindirect information; and at least one of increasing and decreasing theassessed risk if the circumstantial or indirect information is negativeor positive; automatically determining, based on the assessed risk, achange or a setting to at least one element of policy criteria of acyber security policy; automatically recommending, based on the assessedrisk, computer network changes to reduce the assessed risk; providingone or more recommended computer network changes to reduce the assessedrisk, enactment by the entity of at least one of the one or more of therecommended computer network changes to reduce the assessed risk to theentity; determining that the entity has enacted at least a portion ofthe recommended computer network changes, and in response, automaticallyreassessing the risk of a cyber security failure in the computer networkof the entity based on the enacted recommended computer network changes;and dynamically re-determining, based on the reassessed risk of a cybersecurity failure in the computer network of the entity, the change orthe setting to the at least one element of policy criteria of the cybersecurity policy.
 2. The method of claim 1, wherein the cyber securityfailure comprises a cyber attack.
 3. The method of claim 1, wherein thecyber security failure comprises a privacy incident involving sensitiveinformation.
 4. The method of claim 1, wherein the computer agent isfurther configured to perform at least one of collecting informationfrom the computer network of the entity, and analyzing the informationfrom the computer network of the entity.
 5. The method of claim 1,further comprising: based on the assessing of risk of the cyber securityfailure in the computer network of the entity, plotting one or morefeatures of the entity and other members of a peer group of the entity,the plotting being configured to visually illustrate the risk of a cybersecurity failure in the computer network of the entity; and theautomatically recommending of computer network changes being based onthe plotting.
 6. The method of claim 5, wherein the plotting isperformed in a matrix that visually illustrates the risk of a cybersecurity failure in the computer network of the entity.
 7. The method ofclaim 6, further comprising: in response to the determining that theentity has enacted at least a portion of the recommended computernetwork changes, initiating the change or the setting to the at leastone element of policy criteria of the cyber security policy.
 8. Themethod of claim 5, wherein the assessing of risk of the cyber securityfailure in the computer network of the entity comprises assessing, usinga plurality of sophistication elements for the entity, a sophisticationfor the entity with respect to preventing the cyber security failure,the sophistication being one of features of the entity.
 9. The method ofclaim 1, wherein the assessing of risk of the cyber security failure inthe computer network of the entity comprises assessing, using aplurality of motivation elements regarding the entity, a motivation ofan actor to initiate the cyber security failure, the motivation beingone of a plurality of features of the entity.
 10. The method of claim 9,wherein the actor is a hacker.
 11. The method of claim 1, wherein theassessing of risk of a cyber security failure in the computer network ofthe entity comprises: assessing, using a plurality of sophisticationelements for the entity, a sophistication for the entity with respect topreventing the cyber security failure, the sophistication being one of aplurality of features of the entity; and assessing, using a plurality ofmotivation elements regarding the entity, a motivation of an actor toinitiate the cyber security failure, the motivation being another one ofthe features of the entity.
 12. The method of claim 11, furthercomprising calculating a composite score from a motivation score and asophistication score, the motivation score representing the plurality ofmotivation elements, the sophistication score representing the pluralityof sophistication elements.
 13. The method of claim 12, furthercomprising: creating an aggregate risk score of a portfolio of entitiesbased on a plurality of motivation scores including the motivation scoreand a plurality of sophistication scores including the sophisticationscore; and benchmarking over time at least one of the sophisticationscore, the motivation score, the composite score, and the aggregate riskscore.
 14. The method of claim 13, further comprising determining aposition of the entity relative to the aggregate risk score of theportfolio of entities, the portfolio of entities belonging to at leastone of an industry group, a geographic location, a company size, atechnology sector, or any combinations thereof.
 15. The method of claim13, wherein the assessed risk of a cyber security failure in thecomputer network of the entity is a function of the aggregate risk scoreof the portfolio of entities.
 16. The method of claim 13, furthercomprising: comparing the assessed risk of a cyber security failure inthe computer network of the entity to the aggregate risk score; andautomatically generating and recommending additional computer networkchanges to reduce the assessed risk of a cyber security failure in thecomputer network of the entity.
 17. The method of claim 13, furthercomprising identifying clusters of sophistication elements or motivationelements shared between two or more entities of the portfolio ofentities, the clusters of sophistication elements or motivation elementsbeing associated with an increase in the risk of a cyber securityfailure in the computer network of the entity.
 18. The method of claim17, further comprising: identifying additional sophistication elementsor motivation elements for at least one of the two or more entities ofthe portfolio of entities that are not shared with the portfolio ofentities, the additional sophistication elements or motivation elementsbeing associated with another increase in the risk of a cyber securityfailure in the computer network of the entity; and generatingrecommendations for the at least one of the two or more entities of theportfolio of entities that will cause a decrease in the risk of a cybersecurity failure in the computer network of the entity.
 19. The methodof claim 11, further comprising estimating at least one impact to theentity for a simulated cyber security failure, the simulated cybersecurity failure testing the sophistication of the entity and beingaffected by the motivation regarding the entity.
 20. The method of claim19, wherein the estimating of the at least one impact to the entity forthe simulated cyber security failure is dynamically calculated based onthe determining that the entity has enacted at least a portion of therecommended computer network changes.
 21. The method of claim 12,further comprising providing attribution for a change in at least one ofthe composite, motivation, or sophistication scores, the attributionindicating a change in an underlying data set to effect the change inthe at least one of the composite, motivation, or sophistication scores.22. The method of claim 11, wherein the one or more recommended computernetwork changes affect at least one of the motivation and thesophistication of the entity in such a way that the assessed risk of acyber security failure in the computer network of the entity is reduced.23. The method of claim 1, wherein at least one of the one or morerecommended computer network changes is implemented automaticallywithout intervention of the entity.
 24. The method of claim 1, furthercomprising: in response to the determining that the entity has enactedat least a portion of the recommended computer network changes,generating subsequent recommended computer network changes if the riskof a cyber security failure in the computer network of the entity hasnot decreased sufficiently to meet a threshold.
 25. The method of claim1, wherein the cyber security policy is at least one of: a cybersecurity policy from another entity; and a product warranty for firstand/or third party costs that the entity purchases from at least one ofa networking, security product, and services provider.
 26. A system,comprising: a processor; and a memory communicatively coupled with theprocessor, the memory storing instructions which when executed by theprocessor performs a method comprising: assessing risk of a cybersecurity failure in a computer network of an entity, using a computeragent configured to collect information from at least publiclyaccessible Internet elements, wherein the assessing of risk comprises:evaluating the collected information to obtain circumstantial orindirect information regarding the entity, the circumstantial orindirect information having an impact on the risk but the circumstantialor indirect information not specifically referencing the entity; crossreferencing data in the collected information to confirm or infer thatthe entity is referenced in the circumstantial or indirect informationthat is indicative of the entity being referenced in the circumstantialor indirect information; and at least one of increasing and decreasingthe assessed risk if the circumstantial or indirect information isnegative or positive; automatically determining, based on the assessedrisk, a change or a setting to at least one element of policy criteriaof a cyber security policy; automatically recommending, based on theassessed risk, computer network changes to reduce the assessed risk;providing one or more recommended computer network changes to reduce theassessed risk, enactment by the entity of at least one of the one ormore of the recommended computer network changes to reduce the assessedrisk to the entity; determining that the entity has enacted at least aportion of the recommended computer network changes, and in response,automatically reassessing the risk of a cyber security failure in thecomputer network of the entity based on the enacted recommended computernetwork changes; and dynamically re-determining, based on the reassessedrisk of a cyber security failure in the computer network of the entity,the change or the setting to the at least one element of policy criteriaof the cyber security policy.
 27. A method, comprising: receiving anassessment of risk of a cyber security failure in a computer network ofan entity from a computer agent configured to collect information fromat least publicly accessible Internet elements; evaluating the collectedinformation to obtain circumstantial or indirect information that isindicative of the entity; cross referencing data in the collectedinformation to confirm or infer that the entity is referenced in thecircumstantial or indirect information that is indicative of the entitybeing referenced in the circumstantial or indirect information; at leastone of increasing and decreasing the assessment of risk if thecircumstantial or indirect information is negative or positive, whereinthe risk comprises an analysis of the circumstantial or indirectinformation that is indicative of the entity, the circumstantial orindirect information having an impact on the risk but the circumstantialor indirect information not specifically referencing the entity;automatically determining, based on the assessed risk, a change or asetting to at least one element of policy criteria of a cyber securitypolicy; automatically recommending, based on the assessed risk, computernetwork changes to reduce the assessed risk; providing one or morerecommended computer network changes to reduce the assessed risk,enactment by the entity of at least one or more of the recommendedcomputer network changes to reduce the assessed risk to the entity;determining that the entity has enacted at least a portion of therecommended computer network changes, and in response, automaticallyreassessing the risk of a cyber security failure in the computer networkof the entity based on the enacted recommended computer network changes;and dynamically re-determining, based on the reassessed risk of a cybersecurity failure in the computer network of the entity, the change orthe setting to the at least one element of policy criteria of the cybersecurity policy.